Node Aware Sparse Matrix-Vector Multiplication

where A is a sparse N ×N matrix and v is a dense N -dimensional vector. In parallel, the sparse system is often distributed across np processes such that each process holds a contiguous block of rows from the matrix A, and equivalent rows from the vectors v and w, as shown in Figure 1. A common approach is to also split the rows of A on a single process into two groups: an on-process block, containing the columns of the matrix that correspond to vector values stored locally, and an off-process block, containing matrix non-zeros that are associated with vector values that are stored on non-local processes. Therefore, non-zeros in the off-process block of the matrix require vector values to be communicated during each SpMV. The SpMV operation lacks parallel scalability due to large costs associated with communication, specifically in the strong scaling limit of a few rows per process. In-